Moving image composition device, moving image composition method, and information terminal with moving image composition function

ABSTRACT

One of a plurality of items of control-data-for-synthesis (G 2 C) is read from a storage ( 3 ) at a timing based on a moving-image control signal (G 1 C), an image-data-for-synthesis (G 2 G) associated with the read control-data-for-synthesis (G 2 C) is read from the storage ( 3 ) in accordance with the read control-data-for-synthesis (G 2 C), and processing to synthesize one frame of the moving-image data (G 1 G) and the read image-data-for-synthesis (G 2 G) is performed. These processing steps are repeated to produce composite moving-image data (G 3 G).

TECHNICAL FIELD

The present invention relates to a moving-image synthesis device and amoving-image synthesis method for producing composite moving-image databy synthesizing (that is, “moving-image overlaying” or “animationoverlaying”) image-data-for-synthesis such as graphics image data andinput moving-image data in real time, and an information terminalapparatus with a moving-image synthesis function such as a videophoneapparatus.

BACKGROUND ART

A conventional telephone equipped with a camera has a function to selectone of previously stored graphics still images and overlay (that is,“still-image overlay”) the selected graphics still image on a stillimage newly taken by the camera.

Further, Japanese Patent Kokai (Laid-open) Publication No. 2000-175166discloses a videophone apparatus which can send and receive amoving-image and voice. This videophone apparatus has a hide functionwhich prevents the other party from becoming aware of the whereabouts ofa human figure (sender) by overlaying (that is, “still-imageoverlaying”) a still image on the background of the human figure(sender) taken by the image pickup device to make it hard to identifythe background or by replacing the background with a still image.

However, the above-described conventional telephone equipped with acamera can just overlay a still image provided in advance on a stillimage newly taken by the camera, and cannot send or receive amoving-image.

Further, the above-described conventional videophone apparatus has afunction to send and receive a moving-image, but when the hide functionis used, a part of a sent composite image becomes a still image.Therefore, there is a problem that a part of the function to send amoving-image is wasted (that is, the function to send a moving-image isnot put into full use).

Accordingly, an object of the present invention is to provide amoving-image synthesis device, a moving-image synthesis method, and aninformation terminal apparatus with a moving-image synthesis function,which can generate composite moving-image data to provide a fullmoving-image by synthesizing image-data-for-synthesis (that is, “datafor moving-image overlay”) and the input moving-image data in real time.

DISCLOSURE OF INVENTION

A moving-image synthesis device of the present invention includes asynthesis processor which receives a video signal, which includesmoving-image data and a moving-image control signal including displaytiming information of each frame of the moving-image data; and a storagewhich stores data-for-synthesis, which includes a plurality of items ofimage-data-for-synthesis and a plurality of items ofcontrol-data-for-synthesis associated with the plurality of items of theimage-data-for-synthesis. The synthesis processor reads one of theplurality of items of the control-data-for-synthesis from the storage ata timing based on the moving-image control signal; reads theimage-data-for-synthesis associated with the readcontrol-data-for-synthesis from the storage in accordance with the readcontrol-data-for-synthesis; executes processing to synthesize one frameof the moving-image data and the read image-data-for-synthesis; andrepeats these processing steps to produce composite moving-image data.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a configuration of a moving-imagesynthesis device and an information terminal apparatus with amoving-image synthesis function, of a first embodiment of the presentinvention;

FIG. 2 is a block diagram showing a detailed configuration of thedata-for-synthesis storage shown in FIG. 1;

FIG. 3 is a block diagram showing a detailed configuration of thesynthesis controller shown in FIG. 1;

FIG. 4 is a block diagram showing a detailed configuration of the imagesynthesis section shown in FIG. 1;

FIG. 5 is a diagram showing an example of a format of thecontrol-data-for-synthesis stored in the control-data-for-synthesisstorage shown in FIG. 2;

FIG. 6 is a diagram showing types of information stored in eachcontrol-data-for-synthesis item (header) shown in FIG. 5;

FIG. 7 is a diagram showing a relationship between the parameters shownin FIG. 6 and the display screen;

FIG. 8 is a diagram showing an example of a format of theimage-data-for-synthesis stored in the image-data-for-synthesis storageshown in FIG. 2;

FIG. 9 is a diagram showing an example of a relationship between thecontrol-data-for-synthesis and the image-data-for-synthesis;

FIG. 10 is a diagram showing an example of display of the compositeimages in FIG. 9;

FIG. 11 is a diagram showing another example of a relationship betweenthe control-data-for-synthesis and the image-data-for-synthesis;

FIG. 12 is a diagram showing an example of display of the compositeimages in FIG. 11;

FIG. 13 is a block diagram showing a configuration of a moving-imagesynthesis device and an information terminal apparatus with amoving-image synthesis function, of a second embodiment of the presentinvention; and

FIG. 14 is a block diagram showing a configuration of a synthesiscontroller in a moving-image synthesis device of a third embodiment ofthe present invention.

BEST MODE FOR CARRYING OUT THE INVENTION

Embodiments of the present invention will be described below in detailwith reference to the accompanying drawings.

FIRST EMBODIMENT

FIG. 1 is a block diagram showing a configuration of a moving-imagesynthesis device 100 (a device which can implement a moving-imagesynthesis method of the present invention) and an information terminalapparatus 110 with a moving-image synthesis function, of a firstembodiment of the present invention. The information terminal apparatus110 is a cellular phone with a videophone function, for example.

As shown in FIG. 1, the information terminal apparatus 110 includes themoving-image synthesis device 100, an image pickup device 1, adata-for-synthesis input section 2, a CPU 6, a voice input section 7, avoice output section 8, an image display section 9, and a videophoneprocessor 10 which has a function to communicate through a publicnetwork 11.

Further, as shown in FIG. 1, the moving-image synthesis device 100includes a data-for-synthesis storage 3, a synthesis controller 4, andan image synthesis section 5. The synthesis controller 4 and the imagesynthesis section 5 compose a synthesis processor 20 which carries outthe processing to synthesize (that is, “moving-image overlay”)image-data-for-synthesis G2G and moving-image data G1G. The moving-imagesynthesis device 100 is configured by semiconductor integratedcircuitry, for example.

The image pickup device 1 is a camera, for example, and generates avideo signal G1 including moving-image data G1G and moving-image controlsignal G1C accompanying therewith. The generated video signal G1 isseparated into the moving-image data G1G, which is a portion of dataactually displayed as an image, and the moving-image control signal G1C,which is a control data portion, and supplied to the moving-imagesynthesis device 100. The moving-image control signal G1C is supplied tothe CPU 6 as well.

The data-for-synthesis input section 2 is connected to a serialcommunication line (not shown) conforming to a standard such as RS-232Cand USB or connected directly to a CPU bus line (not shown), and inputsdata-for-synthesis G2 to the moving-image synthesis device 100. Thedata-for-synthesis G2 includes image-data-for-synthesis G2G which is apart of data actually displayed as an image, andcontrol-data-for-synthesis G2C which forms sequence data.

The data-for-synthesis storage 3 is an element for storing thedata-for-synthesis G2 supplied from the data-for-synthesis input section2, and will be described later in further detail with reference to FIG.2.

The synthesis controller 4 is an element for controlling the synthesisprocessing of the moving-image data G1G and the image-data-for-synthesisG2G in accordance with the moving-image control signal G1C contained inthe video signal G1 and the control-data-for-synthesis G2C contained inthe data-for-synthesis G2, and will be described later in further detailwith reference to FIG. 3.

The image synthesis section 5 is an element for synthesizing themoving-image data G1G input from the image pickup device 1 and theimage-data-for-synthesis G2G read from the data-for-synthesis storage 3in accordance with the control-data-for-synthesis G2C read from thedata-for-synthesis storage 3 and a control signal from the CPU 6, whichwill be described later, and will be described later in further detailwith reference to FIG. 5.

The CPU (central processing unit) 6 is an arithmetic unit containing aprocessing element such as a microprocessor and peripheral circuitrysuch as a register for outputting a control signal predetermined forimage synthesis at a timing given by the moving-image control signalG1C. The CPU 6 generates a moving-image ratio control signal which isparameter data indicating a ratio of the moving-image in imagesynthesis, an image-for-synthesis ratio control signal which isparameter data indicating a ratio of an image-for-synthesis, and acontrol signal containing a switch control signal CPU_(c), in accordancewith the input moving-image control signal G1C, and outputs the signalsto the image synthesis section 5. In FIG. 1, the moving-image ratiocontrol signal, the image-for-synthesis ratio control signal, amoving-image selection control signal, and an image-for-synthesisselection control signal are collectively expressed as CPU_(p).

The voice input section 7 includes a microphone or the like, andgenerates an audio signal in accordance with the input voice. The voiceoutput section 8 includes a speaker or the like, and outputs a sound inaccordance with the input audio signal. The image display section 9 is aliquid crystal display section, for example, and displays an image basedon the composite moving-image data G3G synthesized by the imagesynthesis section 5, an image based on the moving-image data received bythe videophone processor 10, which will be described later, or the like.

The videophone processor 10 compresses the composite moving-image dataG3G synthesized by the image synthesis section 5 into a videophone imagesignal and sends the videophone image signal to the other party,together with the audio signal input from the voice input section 7. Thevideophone processor 10 also decompresses a signal received from theother party into a videophone image signal and an audio signal andcauses an image to be displayed by the image display section 9 and voiceto be output from the voice output section 8. The public network 11 is acommunication network such as cellular phone channels, wired telephonechannels, and the like.

FIG. 2 is a block diagram showing a detailed configuration of thedata-for-synthesis storage 3 shown in FIG. 1.

A control-data-for-synthesis storage 31 stores thecontrol-data-for-synthesis G2C forming sequence data, out of thedata-for-synthesis G2 input from an input terminal G2 _(in) of thedata-for-synthesis storage 3. The control-data-for-synthesis storage 31outputs the control-data-for-synthesis G2C at the address specified bythe read address information input from the input terminal ADC_(in) ofthe data-for-synthesis storage 3, from an output terminal G2C_(out) ofthe data-for-synthesis storage 3.

An image-data-for-synthesis storage 32 stores theimage-data-for-synthesis G2G, which is a part of data to be displayed asan actual image, out of the data-for-synthesis G2 input from the inputterminal G2 _(in) of the data-for-synthesis storage 3. Theimage-data-for-synthesis storage 32 outputs the image-data-for-synthesisG2G at the address specified by the read address information input fromthe input terminal ADG_(in) of the data-for-synthesis storage 3, from anoutput terminal G2G_(out) of the data-for-synthesis storage 3, and alsooutputs parameter data G2G_(p) such as synthesis conditions and asynthesis ratio parameter of the image-data-for-synthesis G2G, from anoutput terminal G2GP_(out) of the data-for-synthesis storage 3.

FIG. 3 is a block diagram showing a detailed configuration of thesynthesis controller 4 shown in FIG. 1.

The moving-image control signal G1C is a signal accompanying themoving-image data G1G input from the image pickup device 1, and includesa clock, a horizontal synchronization signal, a vertical synchronizationsignal, and a moving-image parameter. A frame reset signal generator 41generates a frame reset signal FRS, which indicates the beginning of aframe, from the moving-image control signal G1C input from an inputterminal G1C_(in), of the synthesis controller 4.

A control-data-for-synthesis read address generator 42 generates acontrol data read address ADC based on the information given by acontrol-data-for-synthesis analyzer 44 at the timing of the frame resetsignal FRS output from the frame reset signal generator 41, and outputsthe address from an output terminal ADC_(out) of the synthesiscontroller 4.

An image-data-for-synthesis read address generator 43 generates an imagedata read address ADG based on the moving-image control signal G1C inputfrom the input terminal G1C_(in) of the synthesis controller 4 and theinformation given by the control-data-for-synthesis analyzer 44, whichwill be described later, and outputs the address from an output terminalADG_(out) of the synthesis controller 4.

The control-data-for-synthesis analyzer 44 analyzes thecontrol-data-for-synthesis G2C (header), which is read at the controldata read address ADC by the control-data-for-synthesis storage 31 andinput from the input terminal G2C_(in) of the synthesis controller 4,and outputs analyzed information to the control-data-for-synthesis readaddress generator 42 and the image-data-for-synthesis read addressgenerator 43.

FIG. 4 is a block diagram showing a detailed configuration of the imagesynthesis section 5 shown in FIG. 1.

A first fader 51 attenuates the amplitude of the moving-image data G1G,which is output from the image pickup device 1 and input through theinput terminal G1G_(in), of the image synthesis section 5. A secondfader 52 attenuates the amplitude of the image-data-for-synthesis G2G,which is output from the data-for-synthesis storage 3 and input throughthe input terminal G2G_(in), of the image synthesis section 5.

An adder 53 sums up the moving-image data G1G attenuated by the firstfader 51 and the image-data-for-synthesis G2G attenuated by the secondfader 52, and outputs composite moving-image data G3G.

A selector 54 selects the moving-image data G1G input from the inputterminal G1G_(in) of the image synthesis section 5, theimage-data-for-synthesis G2G input from the input terminal G2G_(in), ofthe image synthesis section 5, or the composite moving-image data G3Goutput from the adder 53, and outputs the selected data from an outputterminal G3G_(out) of the image synthesis section 5.

A switch 55 switches an image selection control signal to be output tothe selector 54. The switch 55 changes its state in accordance with aswitch control signal CPU_(c), out of the control signals from the CPU6. The switch 55 outputs to the selector 54, either the image selectioncontrol signal CPU_(P) (a moving-image selection control signal, whichis parameter data for selecting a moving-image, and animage-for-synthesis selection control signal, which is parameter datafor selecting an image-for-synthesis), out of the control signals outputfrom the CPU 6 and input through the input terminal CPUP_(in) of theimage synthesis section 5, or the moving-image selection control signaland the image-for-synthesis selection control signal, out of thecontrol-data-for-synthesis G2G_(p), which is parameter data output fromthe data-for-synthesis storage 3 and input through the input terminalG2GP_(in) of the image synthesis section 5.

The switch 56 selects an image ratio control signal output to the firstfader 51 and the second fader 52. The switch 56 changes its state inaccordance with the switch control signal CPU_(c) from the CPU 6. Theswitch 56 outputs to the first fader 51 and the second fader 52, eitherthe image ratio control signal CPU_(p) (a moving-image ratio controlsignal, which is parameter data about a ratio of the moving-image, andan image-for-synthesis ratio control signal, which is parameter dataabout a ratio of the image-for-synthesis), out of the control signalsoutput from the CPU 6 and input through the input terminal CPUP_(in) ofthe image synthesis section 5, or the moving-image ratio control signaland the image-for-synthesis ratio control signal, out of thecontrol-data-for-synthesis G2G_(p) output from the data-for-synthesisstorage 3 and input through the input terminal G2GP_(in) of the imagesynthesis section 5.

The operation of the first embodiment will be described below withreference to FIGS. 1 to 4.

The data-for-synthesis input section 2 was previously supplied with thedata-for-synthesis G2, either through a serial communication means (notshown) conforming to a standard such as RS-232C and USB, or directlyfrom the CPU bus (not shown). The data-for-synthesis G2 input from thedata-for-synthesis input section 2 is supplied to the data-for-synthesisstorage 3. The data-for-synthesis G2 is separated into thecontrol-data-for-synthesis G2C and the image-data-for-synthesis G2G. Ofthe data-for-synthesis G2 input through the input terminal G2 _(in) tothe data-for-synthesis storage 3, the control-data-for-synthesis G2C isstored in the control-data-for-synthesis storage 31, and theimage-data-for-synthesis G2G is stored in the image-data-for-synthesisstorage 32.

On the other hand, the moving-image data G1G in the video signal G1output from the image pickup device 1 is input to the image synthesissection 5. At the same time, the moving-image control signal G1C, whichis a control signal such as a clock, a horizontal synchronizationsignal, and a vertical synchronization signal, of the video signal G1 isinput to the synthesis controller 4 and the CPU 6.

The moving-image control signal G1C is input through the input terminalG1C_(in), of the synthesis controller 4 to the frame reset signalgenerator 41. The frame reset signal generator 41 catches the beginningof the vertical synchronization signal included in the moving-imagecontrol signal G1C to generate a frame reset signal FRS, for example.Therefore, the frame reset signal FRS becomes a signal representing thebeginning of each frame. The frame reset signal FRS generated by theframe reset signal generator 41 is input to thecontrol-data-for-synthesis read address generator 42. At a timing whenthe frame reset signal FRS is input, the control-data-for-synthesis readaddress generator 42 outputs the control data read address ADC of thedata-for-synthesis G2, through the output terminal ADC_(out) of thesynthesis controller 4 to the data-for-synthesis storage 3. The controldata read address ADC is generated on the basis of information providedby the control-data-for-synthesis analyzer 44 after analyzing the header(control-data-for-synthesis) of the data-for-synthesis G2 read in theprevious frame.

The control data read address ADC output from the synthesis controller 4is input through the input terminal ADC_(in) to thecontrol-data-for-synthesis storage 31 in the data-for-synthesis storage3. The control-data-for-synthesis storage 31 reads a stored header(control-data-for-synthesis) in accordance with the input control dataread address ADC, and outputs the header through the output terminalG2C_(out) of the data-for-synthesis storage 3 to the synthesiscontroller 4. The data format and the header configuration of the header(control-data-for-synthesis) stored in the control-data-for-synthesisstorage 31 will be described later with reference to FIG. 5 and FIG. 6.

The header (control-data-for-synthesis) read by thecontrol-data-for-synthesis storage 31 in the data-for-synthesis storage3 is input through the input terminal G2C_(in), to thecontrol-data-for-synthesis analyzer 44 in the synthesis controller 4. Ofthe information of analysis results of the control-data-for-synthesisanalyzer 44, the leading address of the header portion of the nextdata-for-synthesis (address obtained from “the pointer information 3116pointing at the next control-data-for-synthesis (header)” in FIG. 6,which will be described later) is output to thecontrol-data-for-synthesis read address generator 42, and theinformation of the other parameters (information 3111 to 3115 and 3117in FIG. 6, which will be described later, and the like) is output to theimage-data-for-synthesis read address generator 43.

In accordance with the moving-image control signal G1C (a clock,horizontal and vertical synchronization signals, and the like) inputfrom the image pickup device 1 through the input terminal G1C_(in) ofthe synthesis controller 4 and the parameter information of analysisresults of the control-data-for-synthesis analyzer 44, theimage-data-for-synthesis read address generator 43 determines whetherthe data-for-synthesis storage 3 should obtain theimage-data-for-synthesis G2G from the data-for-synthesis input section2, so that the image synthesis section 5 can synthesize the moving-imagedata G1G from the image pickup device 1 and the image-data-for-synthesisG2G from the data-for-synthesis storage 3 in accordance with theparameters analyzed by the control-data-for-synthesis analyzer 44. Ifthe image-data-for-synthesis G2G should be obtained, an acquisitionrequest is output together with the image-data-for-synthesis readaddress ADG.

In the data-for-synthesis storage 3, the image-data-for-synthesisstorage 32 receives the image-data-for-synthesis read address ADG andthe data-for-synthesis acquisition request from the synthesis controller4 through the input terminal ADG_(in). The image-data-for-synthesis G2Gis read from the address specified by the image-data-for-synthesis readaddress ADG of the image-data-for-synthesis storage 32, and outputthrough the output terminal G2G_(out) to the image synthesis section 5.If each pixel in the image-data-for-synthesis G2G has a data formatincluding synthesis ratio parameters in the image synthesis section 5(the moving-image ratio control signal and the image-for-synthesis ratiocontrol signal), image selection parameters (the moving-image selectioncontrol signal and the image-for-synthesis selection control signal),and synthesis condition parameters and the like, these parameters G2GPare output through the output terminal G2GP_(out) of thedata-for-synthesis storage 3 to the image synthesis section 5.

As shown in FIG. 4, in the image synthesis section 5, the moving-imageselection control signal and the image-for-synthesis selection controlsignal of the parameters (control-data-for-synthesis) G2GP are inputthrough the input terminal G2GP_(in) to the input portion 55 b of theswitch 55, and the moving-image ratio control signal and theimage-for-synthesis ratio control signal of the parameters G2GP areinput to the input portion 56 b of the switch 56. The moving-imageselection control signal and the image-for-synthesis selection controlsignal from a register or the like of the CPU 6 are input through theinput terminal CPUP_(in) to the input portion 55 a of the switch 55; andthe moving-image ratio control signal and the image-for-synthesis ratiocontrol signal from a register or the like of the CPU 6 are inputthrough the input terminal CPUP_(in) to the input portion 56 a of theswitch 56. The CPU 6 supplies a control signal CPU_(c) for selectingwhether a control signal of either the input portion 55 a or the inputportion 55 b is output to the output portion 55 c of the switch 55,through the input terminal CPUP_(in) to the control terminal 55 _(s) ofthe switch 55. The CPU 6 also supplies a control signal CPU_(c) forselecting whether a control signal of either the input portion 56 a orthe input portion 56 b of the switch 56 is output to the output portion56 c, through the input terminal CPUP_(in) to the control terminal 56_(s) of the switch 56.

Further, as shown in FIG. 4, the moving-image data G1G is supplied fromthe image pickup device 1, through the input terminal G1G_(in), to theimage synthesis section 5. Further, the image-data-for-synthesis G2G issupplied from the image-data-for-synthesis storage 32 in thedata-for-synthesis storage 3, through the input terminal G2G_(in), tothe image synthesis section 5. The moving-image data G1G input from theinput terminal G1G_(in) of the image synthesis section 5 is supplied tothe first fader 51 and the selector 54. Further, theimage-data-for-synthesis G2G input from the input terminal G2G_(in) ofthe image synthesis section 5 is supplied to the second fader 52 and theselector 54. The moving-image data G1G input to the first fader 51 islevel-adjusted (attenuated) by the first fader 51 and supplied to theadder 53. Similarly, the image-data-for-synthesis G2G input to thesecond fader 52 is level-adjusted (attenuated) by the second fader 52and supplied to the adder 53.

The first fader 51 and the second fader 52 are controlled incombination, and a control signal is output from the switch 56. Theadder 53 sums up the output of the first fader 51 and the output of thesecond fader 52, and outputs the results to the selector 54. Theselector 54 selects and outputs the moving-image data G1G input from theinput terminal G1G_(in), the image-data-for-synthesis G2G input from theinput terminal G2G_(in), or the composite moving-image data G3G of themoving-image data G1G and the image-data-for-synthesis G2G output fromthe adder 53, in accordance with the output of the switch 55. The outputof the selector 54 is supplied through the output terminal G3G_(out) tothe videophone processor 10.

The videophone processor 10 compresses data from the image synthesissection 5 in a method defined by a standard such as an ITU-T(International Telecommunications Union-TelecommunicationStandardization Sector) recommendation H.263 and an MPEG4 standard;carries out voice compression processing for the audio data input fromthe voice input section 7; and sends the compressed data to the publicnetwork 11, using a videophone communication procedure. Conversely, thecompressed data from the public network 11 is separated into audio dataand image data by the videophone processor 10, and is decompressed whennecessary. The audio data is output to the voice output section 8, andthe image data is output to the image display section 9.

The format of the control-data-for-synthesis (header) G2C of thecontrol-data-for-synthesis storage 31 in the data-for-synthesis storage3 will next be described.

FIG. 5 is a diagram showing an example of a format of thecontrol-data-for-synthesis stored in the control-data-for-synthesisstorage 31 shown in FIG. 2.

The data format shown in FIG. 5 is composed of N items of thecontrol-data-for-synthesis (headers) numbered by 0 to N−1 (N is apositive integer), denoted by reference numerals 311 to 314. The headerrepresents the synthesis state of each frame when the image is varied ina given sequence.

FIG. 6 is a diagram showing types of information stored in each item ofthe control-data-for-synthesis (header) shown in FIG. 5 (for example,control-data-for-synthesis 311).

The header shown in FIG. 6 is composed of parameters such as “ahorizontal position 3111 of image-for-synthesis”, “a vertical position3112 of image-for-synthesis”, “a horizontal size 3113 ofimage-for-synthesis”, “a vertical size 3114 of image-for-synthesis”,“pointer information 3115 pointing at the currentimage-data-for-synthesis”, “pointer information 3116 pointing at thenext control-data-for-synthesis (header)”, and “a repetition count 3117of current image-for-synthesis.” Further, the structure of thecontrol-data-for-synthesis G2C (header) is not limited to thisstructure. The structure of the control-data-for-synthesis may store theparameters in a different order, for example.

FIG. 7 is a diagram showing a relationship between the parameterscomposing the header shown in FIG. 6 and the display screen of a singleframe of a composite moving-image.

The scanning of one frame of the taken moving-image 91 shown in FIG. 7starts from the top left corner and ends in the bottom right corner ofFIG. 7 (that is, horizontal scanning is carried out from left to rightin FIG. 7, and vertical scanning is carried out from top to bottom inFIG. 7). “The horizontal position 3111 of image-for-synthesis”represents a position (horizontal position) where the top left-end pixelof the image-data-for-synthesis G2G of the image-for-synthesis 92 issuperimposed (OV_POS_H in the figure), when the position (horizontalposition) of the top left-end pixel of the moving-image data G1G is usedas the reference (0). “The vertical position 3112 ofimage-for-synthesis” represents a position (vertical position) where thetop left-end pixel of the image-data-for-synthesis G2G of theimage-for-synthesis 92 is superimposed (OV_POS_V in the figure), whenthe position (vertical position) of the top left-end pixel of themoving-image data G1G is used as the reference (0). “The horizontal size3113 of image-for-synthesis” represents the horizontal size (OV_HSIZE inthe figure) of the image-data-for-synthesis G2G. “The vertical size 3114of image-for-synthesis” represents the vertical size (OV_VSIZE in thefigure) of the image-data-for-synthesis G2G.

“The pointer information 3115 pointing at the currentimage-data-for-synthesis” of the header represents theimage-data-for-synthesis G2G corresponding to the header. Based on thepointer information 3115 pointing at the currentimage-data-for-synthesis, the address where the first data of theimage-data-for-synthesis G2G is stored in the data-for-synthesis storage3 can be obtained. Further, “the pointer information 3116 pointing atthe next control-data-for-synthesis (header)” of the header points atthe next header data. Based on the pointer information 3116 pointing atthe next control-data-for-synthesis (header), the address where thefirst data of the next header data is stored can be obtained. Theparameter for determining the header data to be referenced next(indicating the image-for-synthesis to be referenced next, based on theparameter of “the leading address 3115 of the currentimage-data-for-synthesis”) is important when a plurality of items of theheader data form a sequence.

“The repetition count 3117 of current image-for-synthesis” of the headerindicates a repetition count of the image-data-for-synthesis G2Gcurrently being used for synthesis processing of theimage-data-for-synthesis G2G and the moving-image data G1G (number ofrepetition).

The format of the image data stored in the image-data-for-synthesisstorage 32 of the data-for-synthesis storage 3 will next be described.FIG. 8 is a diagram showing an example of a format of theimage-data-for-synthesis G2G stored in the image-data-for-synthesisstorage 32.

The data format shown in FIG. 8 is composed of M items of theimage-data-for-synthesis numbered by 0 to M−1 (M is a positive integer),denoted by reference numerals 321 to 324. The image-data-for-synthesisG2G of M frames are stored here. The data of each frame of theimage-data-for-synthesis G2G do not need to be of the full size of oneframe. Just the image data corresponding to the size of “the horizontalsize 3113 of image-for-synthesis” multiplied by “the vertical size 3114of image-for-synthesis” should be provided as described above. Acomposite image provided by “the horizontal size 3113 ofimage-for-synthesis” and “the vertical size 3114 ofimage-for-synthesis”, which are control parameters related to the sizeof the composite image, and “the horizontal position 3111 ofimage-for-synthesis” and “the vertical position 3112 ofimage-for-synthesis”, which are control parameters related to theposition of the composite image becomes like one denoted by thereference numeral 92 shown in FIG. 7.

FIG. 9 is a diagram showing an example of a relationship between thecontrol-data-for-synthesis G2C and the image-data-for-synthesis G2G, andFIG. 10 is a diagram showing an example of display of the compositeimages formed in FIG. 9. Further, FIG. 11 is a diagram showing anotherexample of a relationship between the control-data-for-synthesis G2C andthe image-data-for-synthesis G2G, and FIG. 12 is a diagram showing anexample of display of the composite images formed in FIG. 11.

FIG. 9 and FIG. 10 show a case where “the repetition count 3117 ofcurrent image-for-synthesis” of each header is zero (that is, a case ofdisplaying the image-data-for-synthesis item G2G in one frame anddisplaying the next image-data-for-synthesis item G2G in the nextframe). FIG. 11 and FIG. 12 show a case where “the repetition count 3117of current image-for-synthesis” of the second header G2C₂ from the topis one (that is, a case of displaying an image-data-for-synthesis G2G intwo consecutive frames).

The example shown in FIG. 9 and FIG. 10 will now be described. Thesequence of this example is specified as follows:

-   The header G2C₁ is read from the control-data-for-synthesis storage    31, and consequently, the image-data-for-synthesis G2G₁ is read from    the image-data-for-synthesis storage 32;-   The header G2C₂ is read from the control-data-for-synthesis storage    31, and consequently, the image-data-for-synthesis G2G₃ is read from    the image-data-for-synthesis storage 32;-   The header G2C₃ is read from the control-data-for-synthesis storage    31, and consequently, the image-data-for-synthesis G2G₁ is read from    the image-data-for-synthesis storage 32;-   The similar processing is repeated; and-   The header G2C_(N) is read from the control-data-for-synthesis    storage 31, and consequently, the image-data-for-synthesis G2G₂ is    read from the image-data-for-synthesis storage 32. This example is    the case where “the repetition count 3117 of current    image-for-synthesis” of each header is zero.

In FIG. 9, when t=t₁, the header G2C₁ is read from thecontrol-data-for-synthesis storage 31 before the moving-image data G1G₁is input from the image pickup device 1. In accordance with the contentsof the header G2C₁, the image-data-for-synthesis G2G₁ is read from theimage-data-for-synthesis storage 32 at a timing based on the inputtiming of the moving-image data G1G₁, and the image synthesis section 5produces the composite image G3G₁ and outputs the composite image G3G₁to the videophone processor 10. To be more precise, switching for eachpixel is carried out in accordance with a control signal to the selector54 in the image synthesis section 5 so that the moving-image data G1G₁,the image-data-for-synthesis G2G₁, or the composite image G3G₁ formedfrom the moving-image data G1G₁ and the image-data-for-synthesis G2G₁ isselectively output to the videophone processor 10.

When t=t₂, the header G2C₂ is read from the control-data-for-synthesisstorage 31 before the moving-image data G1G₂ is input from the imagepickup device 1. The reason why the header G2C₂ is read is because “thepointer information 3116 pointing at the next control-data-for-synthesis(header)” in the header G2C₁ points at the header G2C₂. Accordingly,there is no need to read the headers G2C in the order in which theheaders G2C were stored in the control-data-for-synthesis storage 31.Further, there is no need to store the headers G2C in the order of thesequence. The image-data-for-synthesis G2G₃ is read from theimage-data-for-synthesis storage 32 at a timing based on the inputtiming of the moving-image data G1G₂, in accordance with the contents ofthe header G2C₂, and then the image synthesis section 5 produces thecomposite image G3G₂ and outputs the composite image G3G₂ to thevideophone processor 10.

When t=t₃, the header G2C₃ is read from the control-data-for-synthesisstorage 31 before the moving-image data G1G₃ is input from the imagepickup device 1. The reason why the header G2C₃ is read is because “thepointer information 3116 pointing at the next control-data-for-synthesis(header)” in the header G2C₂ points at the header G2C₃. In accordancewith the contents of the header G2C₃, the image-data-for-synthesis G2G₁is read from the image-data-for-synthesis storage 32, at a timing basedon the input timing of the moving-image data G1G₃, and then the imagesynthesis section 5 produces the composite image G3G₃ and outputs thecomposite image G3G₃ to the videophone processor 10. The header G2C₃references the image-data-for-synthesis G2G₁, as in the case of theheader G2C₁. If “the horizontal size 3113 of image-for-synthesis” and“the vertical size 3114 of image-for-synthesis” are the same, the sameimage-data-for-synthesis G2G may be referenced.

When t=t_(N), the header G2C_(N) is read from thecontrol-data-for-synthesis storage 31 before the moving-image dataG1G_(N) is input from the image pickup device 1. The reason why theheader G2C_(N) is read is because “the pointer information 3116 pointingat the next control-data-for-synthesis (header)” in the previous headerG2C points at the header G2C_(N). The previous header G2C is notnecessarily the (N−1)−th header G2C stored in thecontrol-data-for-synthesis storage 31. The image-data-for-synthesis G2G₂is read from the image-data-for-synthesis storage 32 in accordance withthe contents of the header G2C_(N) at a timing based on the input timingof the moving-image data G1G_(N), and the image synthesis section 5produces the composite image G3G_(N) and outputs the composite imageG3G_(N) to the videophone processor 10.

When t=t_(N+1), the header G2C₁ is read from thecontrol-data-for-synthesis storage 31 before the moving-image dataG1G_(N+1) is input from the image pickup device 1. The reason why theheader G2C₁ is read is because “the pointer information 3116 pointing atthe next control-data-for-synthesis (header)” in the header G2C_(N)points at the header G2C₁. Even if the control-data-for-synthesisstorage 31 stores N headers G2C, all the N headers do not have to beused. If “the pointer information 3116 pointing at the nextcontrol-data-for-synthesis (header)” of the (N−1)−th header G2C is setto point at the header G2C₁, the N−th header G2C will never be read. Theimage-data-for-synthesis G2G₁ is read from the image-data-for-synthesisstorage 32 at a timing based on the input timing of the moving-imagedata G1G_(N+1), in accordance with the contents of the header G2C₁, andthe image synthesis section 5 produces the composite image G3G_(N+1) andoutputs the composite image G3G_(N+1) to the videophone processor 10.

In accordance with the sequence formed by the headers from G2C₁ toG2C_(N), the image-data-for-synthesis G2G input to the image synthesissection 5 are further updated, and the composite image data G3G areproduced from the moving-image data G1G and the image-data-for-synthesisG2G.

Next, an example shown in FIG. 11 and FIG. 12 will be described. Thesequence of this example is specified as follows:

-   The header G2C₁ is read from the control-data-for-synthesis storage    31, and consequently, the image-data-for-synthesis G2G₁ is read from    the image-data-for-synthesis storage 32;-   The header G2C₂ is read from the control-data-for-synthesis storage    31, and consequently, the image-data-for-synthesis G2G₃ is read from    the image-data-for-synthesis storage 32:-   The header G2C₃ is read from the control-data-for-synthesis storage    31, and consequently, the image-data-for-synthesis G2G₁ is read from    the image-data-for-synthesis storage 32;-   The similar processing is repeated; and-   The header G2C_(N) is read from the control-data-for-synthesis    storage 31, and consequently, the image-data-for-synthesis G2G₂ is    read from the image-data-for-synthesis storage 32. In this example,    “the repetition count 3117 of current image-for-synthesis” of the    header G2C₂ is one.

When t=t₁, the header G2C₁ is read from the control-data-for-synthesisstorage 31 before the moving-image data G1G₁ is input from the imagepickup device 1. In accordance with the contents of the header G2C₁, theimage-data-for-synthesis G2G₁ is read from the image-data-for-synthesisstorage 32 at a timing based on the input timing of the moving-imagedata G1G₁, and the image synthesis section 5 produces the compositeimage G3G₁ and outputs the composite image G3G₁ to the videophoneprocessor 10. To be more precise, switching for each pixel is carriedout by a control signal to the selector 54 in the synthesis section 5,and the moving-image data G1G₁, the image-data-for-synthesis G2G₁, orthe composite image G3G₁ of the moving-image data G1G₁ and theimage-data-for-synthesis G2G₁ is selectively output to the videophoneprocessor 10.

When t=t₂, the header G2C₂ is read from the control-data-for-synthesisstorage 31 before the moving-image data G1G₂ is input from the imagepickup device 1. The reason why the header G2C₂ is read is just because“the pointer information 3116 pointing at the nextcontrol-data-for-synthesis (header)” in the header G2C₁ points at theheader G2C₂. Accordingly, there is no need to read the headers G2C inthe order in which the headers were stored in thecontrol-data-for-synthesis storage 31. Further, there is no need tostore the headers G2C in the order of the sequence. In accordance withthe contents of the header G2C₂, the image-data-for-synthesis G2G₃ isread from the image-data-for-synthesis storage 32 at a timing based onthe input timing of the moving-image data G1G₂, and the image synthesissection 5 produces the composite image G3G₂ and outputs the compositeimage G3G₂ to the videophone processor 10.

When t=t₃, the header G2C₂ is read from the control-data-for-synthesisstorage 31 before the moving-image data G1G₃ is input from the imagepickup device 1. The reason why the header G2C₂ is read is because “therepetition count 3117 of current image-for-synthesis” in the header G2C₂read when t=t₂ is set to one. In another configuration, if “therepetition count 3117 of current image-for-synthesis” in the header G2C₂read when t=t₂ is set to one, the header G2C is not read from thecontrol-data-for-synthesis storage 31 at the next timing (when t=t₃),and the contents of the header G2C when t=t₂ may be continued to beheld. In accordance with the contents of the header G2C₂, theimage-data-for-synthesis G2G₃ is read from the image-data-for-synthesisstorage 32 at a timing based on the input timing of the moving-imagedata G1G₃, and the image synthesis section 5 produces and outputs thecomposite image G3G₃ to the videophone processor 10.

When t=t₄, the header G2C₃ is read from the control-data-for-synthesisstorage 31 before the moving-image data G1G₄ is input from the imagepickup device 1. The reason why the header G2C₃ is read is because “thepointer information 3116 pointing at the next control-data-for-synthesis(header)” in the header G2C₂ points at the header G2C₃. In accordancewith the contents of the header G2C₃, the image-data-for-synthesis G2G₁is read from the image-data-for-synthesis storage 32 at a timing basedon the input timing of the moving-image data G1G₄, and the imagesynthesis section 5 produces the composite image G3G₄ and outputs thecomposite image G3G₄ to the videophone processor 10. The header G2C₃references the image-data-for-synthesis G2G₁ as in the case of theheader G2C₁. If “the horizontal size 3113 of image-for-synthesis” and“the vertical size 3114 of image-for-synthesis” are the same, the sameimage-data-for-synthesis G2G may be referenced.

When t=t_(N+1), the header G2C_(N) is read from thecontrol-data-for-synthesis storage 31 before the moving-image dataG1G_(N+1) is input from the image pickup device 1. The reason why theheader G2C_(N) is read is because “the pointer information 3116 pointingat the next control-data-for-synthesis (header)” in the previous headerG2C points at the header G2C_(N). There is no need of the previousheader G2C being the (N−1)−th header G2C stored in thecontrol-data-for-synthesis storage 31. In accordance with the contentsof the header G2C_(N), the image-data-for-synthesis G2G₂ is read fromthe image-data-for-synthesis storage 32 at a timing based on the inputtiming of the moving-image data G1G_(N+1), and the image synthesissection 5 produces the composite image G3G_(N+1) and outputs thecomposite image G3G_(N+1) to the videophone processor 10.

When t=t_(N+2), the header G2C₁ is read from thecontrol-data-for-synthesis storage 31 before the moving-image dataG1G_(N+2) is input from the image pickup device 1. The reason why theheader G2C₁ is read is because “the pointer information 3116 pointing atthe next control-data-for-synthesis (header)” in the header G2C_(N)points at the header G2C₁. There is no need to use all of the N headersG2C even if the N headers are stored in the control-data-for-synthesisstorage 31. If “the pointer information 3116 pointing at the nextcontrol-data-for-synthesis (header)” of the (N−1)−th header G2C pointsat the header G2C₁, the N−th header G2C will not be read. In accordancewith the contents of the header G2C₁, the image-data-for-synthesis G2G₁is read from the image-data-for-synthesis storage 32 at a timing basedon the input timing of the moving-image data G1G_(N+2), and the imagesynthesis section 5 produces the composite image G3G_(N+2) and outputsthe composite image G3G_(N+2) to the videophone processor 10.

The number of headers G2C stored in the control-data-for-synthesisstorage 31 and the number of items of image-data-for-synthesis G2Gstored in the image-data-for-synthesis storage 32 (the number of frames)will now be described.

The number of headers G2C and the number of items ofimage-data-for-synthesis G2G are not subject to important constraints.However, from the point of view of the memory size, it is practicallyvaluable that the number of the headers G2C is not less than the numberof items of the image-data-for-synthesis G2G. In that case, byspecifying the same image-data-for-synthesis G2G for a still (orslow-moving) frame (scene), the memory size can be reduced and a longsequence can be specified. If the same image-data-for-synthesis G2G canbe specified for an A frame and a B frame (the tenth frame and thetwentieth frame, for example) in the same sequence (if the tenth frameand the twentieth frame can use the same image-data-for-synthesis), thememory size can be reduced further and a further long sequence can bespecified. Of course, all the stored headers G2C and items ofimage-data-for-synthesis G2G need not be used. A necessary header G2Cand a necessary image-data-for-synthesis G2G can be used in a desiredorder to form a sequence.

Further, “the repetition count 3117 of current image-for-synthesis” canbe separately specified for each frame. This is effective when the speedof motion should be varied in a sequence by using a small number ofitems of the image-data-for-synthesis G2G. This can also reduce thememory size.

Because the image synthesis section 5 forms a composite image by readingthe image-data-for-synthesis G2G from the image-data-for-synthesisstorage 32 in synchronization with the input of the moving-image dataG1G from the image pickup device 1, the composite image can be producedwithout interruption within a frame.

If the frame rate of the moving-image data G1G input from the imagepickup device 1 is high (if a large number of frames are input in onesecond), time-varying of the image-data-for-synthesis G2G can beappropriately adjusted by setting “the repetition count 3117 of currentimage-for-synthesis” (setting the repetition count to high, forexample).

As has been described above, the moving-image synthesis device 100 ofthe first embodiment can automatically synthesize theimage-data-for-synthesis G2G forming a frame of a moving-image and themoving-image data G1G in synchronization with the moving-image data G1Gfrom the image pickup device 1 by setting a synthesis sequence in thecontrol-data-for-synthesis storage 31 in the data-for-synthesis storage3, so that a moving composite image can be formed without putting a loadon the CPU such as when the CPU executes software.

Further, the images are synthesized in synchronization with frames, sothat the frames will not be output midway through synthesis processing.Therefore, good composite moving-images can be obtained.

Furthermore, by causing the headers G2C and the image-data-for-synthesisG2G forming a sequence to be related with each other, and a plurality ofsequences can share the image-data-for-synthesis G2G, so that the amountof image-data-for-synthesis G2G to be provided can be suppressed.

Moreover, a repetition count can be specified separately for each frame,so that a moving-image overlay with varying speed can be carried out.

In addition, an image formed by overlaying an animated overlay image ona taken image can be sent through a videophone channel, and many otherapplications can be found in information terminal apparatuses,especially in cellular phones and the like.

Further, the moving-image synthesis device 100 of the first embodimentholds the size of the image-data-for-synthesis G2G and the imagesynthesis position of the image-data-for-synthesis G2G corresponding tothe moving-image data G1G output from the image pickup device 1 andother moving-image input stage as the data of the header G2C, so thatthe image-data-for-synthesis G2G and the moving-image data output fromthe moving-image input stage can be synthesized in a given position andin a given size.

Furthermore, the moving-image synthesis device 100 of the firstembodiment holds a pointer pointing at the address where theimage-data-for-synthesis G2G of the current frame is stored and apointer pointing at the data of the header G2C of the next frame, as thedata of the header G2C, so that the items of image-data-for-synthesisG2G and the headers G2C can be stored in given addresses and in givenorders, in the data-for-synthesis storage.

Moreover, the moving-image synthesis device 100 of the first embodimentholds the repetition count of the image-data-for-synthesis G2G of thecurrent frame as the data of the header G2C, so that the sequence formedwith the image-data-for-synthesis G2G can be specified to an optimumspeed if the moving-image data G1G is input from the moving-image inputstage at a high frame rate.

In addition, the moving-image synthesis device 100 of the firstembodiment can set the number of frames of the image-data-for-synthesisG2G to be stored in the data-for-synthesis storage to the same number asor a lower number than the number of frames of the data of the headerG2C, so that a long sequence of moving-image can be synthesized with asmall amount of image-data-for-synthesis G2G, and the memory can beefficiently used in relation to the length of the sequence.

SECOND EMBODIMENT

FIG. 13 is a block diagram showing a configuration of a moving-imagesynthesis device 200 (a device which can implement the moving-imagesynthesis method of the present invention) and an information terminalapparatus 210 with a moving-image synthesis function, including themoving-image synthesis device 200, of a second embodiment of the presentinvention. Those structures of the second embodiment in FIG. 13 that areidentical to or correspond to structures of the first embodiment areassigned identical symbols.

The information terminal apparatus 210 of the second embodiment shown inFIG. 13 is a composite moving-image display apparatus which cansynthesize an image-for-synthesis and an input video signal in real timeand can display a composite moving-image.

The information terminal apparatus 210 of the second embodiment shown inFIG. 13 differs from the apparatus of the first embodiment shown in FIG.1 in the following points: the image pickup device 1 is replaced by avideo signal input section 12; the voice input section 7, the voiceoutput section 8, and the videophone processor 10 are eliminated; andthe image display section 9 is connected directly to the image synthesissection 5.

The video signal input section 12 is connected to a video input terminal(not shown) or is connected directly to a CPU bus (not shown) or thelike, and receives the input of a digital video signal and a decodesignal of MPEG4 or the like (collectively referred to as a digital videosignal).

In the second embodiment, a digital video signal input to the videosignal input section 12 is separated into moving-image data G1G and amoving-image control signal G1C. The moving-image data G1G is input tothe image synthesis section 5, and the moving-image control signal G1Cis input to the synthesis controller 4 and the CPU 6. The imagesynthesis section 5 and the synthesis controller 4 perform the sameprocessing as in the first embodiment, and form the compositemoving-image data G3G by synthesizing the moving-image data G1G and theimage-data-for-synthesis G2G.

The moving-image synthesis device 200 or the information terminalapparatus 210 having a moving-image synthesis function of the secondembodiment can automatically synthesize the image-data-for-synthesis G2Gforming a frame of a moving-image and the moving-image data G1G, whichis a digital video signal from the video signal input section 12, insynchronization with the moving-image data G1G by setting a synthesissequence in the control-data-for-synthesis storage 31 in thedata-for-synthesis storage 3, so that a moving composite image can beformed without putting a load on the CPU such as when the CPU executessoftware.

Further, the images are synthesized in synchronization with frames, sothat the frames will not be output midway through synthesis processing.Therefore, good composite moving-images can be obtained.

Furthermore, by causing the headers G2C and the image-data-for-synthesisG2G forming a sequence to be related with each other, and a plurality ofsequences can share the image-data-for-synthesis G2G, so that the amountof image-data-for-synthesis G2G to be provided can be suppressed.

Moreover, a repetition count can be specified separately for each frame,so that a moving-image overlay with varying speed can be carried out.

Except for the above described respects, the second embodiment is thesame as the first embodiment.

THIRD EMBODIMENT

FIG. 14 is a block diagram of a synthesis controller 4 a in amoving-image synthesis device (a device which can implement themoving-image synthesis method of the present invention) of a thirdembodiment of the present invention.

The moving-image synthesis device of the third embodiment differs fromthe moving-image synthesis device of the first embodiment shown in FIG.1 in that the synthesis controller 4 of the first embodiment is replacedby the synthesis controller 4 a shown in FIG. 14.

The synthesis controller 4 a of the third embodiment shown in FIG. 14differs from the synthesis controller 4 of the first embodiment shown inFIG. 3 in that the synthesis controller 4 a has a frame rate detector45, and the control-data-for-synthesis analyzer 44 of the firstembodiment is replaced by the control-data-for-synthesis analyzer 46 ofthe third embodiment. The control-data-for-synthesis analyzer 46receives a frame rate FRT output from the frame rate detector 45 andcontrols the control-data-for-synthesis read address generator 42 andthe image-data-for-synthesis read address generator 43.

The control-data-for-synthesis analyzer 46 controls “the repetitioncount 3117 of current image-for-synthesis” included in the informationof the header G2C read from the control-data-for-synthesis storage 31 inthe data-for-synthesis storage 3, through the input terminal G2C_(in),of the synthesis controller 4 a.

Suppose that the frame rate of the moving-image data G1G output from theimage pickup device 1 is N*M (N and M are positive integers), ascalculated by the frame rate detector 45. For example, suppose that N is5, M is 3, and the frame rate is 15 fps (frames per second). Here, “therepetition count 3117 of current image-for-synthesis”, which is includedin the parameters of the header G2C stored in thecontrol-data-for-synthesis storage 31 within the data-for-synthesisstorage 3, is set to M (=3) beforehand. This value is prepared so thatthe motion (frame rate) of the composite moving-image is optimized(becomes effective) at a frame rate of N*M. Here, the optimum andeffective motion of the composite image is at 5 fps.

Then, suppose that the operating environment changes. For example,suppose that the ambient light becomes low and the frame rate of themoving-image data G1G output from the image pickup device 1 ismultiplied by L/M (L is a positive integer) to be set to N*L. When L=1,for example, the frame rate after the change is 5 fps. The frame ratedetector 45 provides a calculation result of N*L and outputs theinformation to the control-data-for-synthesis analyzer 46. Whenreceiving the information of a frame rate of N*L, thecontrol-data-for-synthesis analyzer 46 actually multiplies therepetition count by L/M and changes the repetition count to L.

Because L=1, the repetition count is 1 here. The actual rate of theimage-data-for-synthesis G2G is 5 fps, and the rate of the originalimage is kept at the optimum level (effective level). In other words,“the repetition count 3117 of current image-for-synthesis” included inthe header G2C is set in correspondence with the reference frame rate(N*M in the example given above). If the frame rate of the moving-imagedata G1G changes, the control-data-for-synthesis analyzer 46automatically calculates in accordance with the frame rate informationfrom the frame rate detector 45, and the control data read addressgenerator 43 and the image-data-for-synthesis read address generator 43are controlled accordingly.

As has been described above, the moving-image synthesis device of thethird embodiment has the frame rate detector 45 in the synthesiscontroller 4 a and automatically changes the actual compositemoving-image repetition count in accordance with the calculated framerate, so that the optimum and effective motion of the original image canbe maintained, and consequently, an optimum and efficient compositemoving-image can be obtained.

Further, the third embodiment has been described above, supposing thatthe frame rate of the moving-image data G1G output from the taken image1 changes automatically. The frame rate can be changed intentionally ascontrolled by the CPU or the like. In that case, the frame rate need notbe calculated by the frame rate detector 45. Accordingly, the sameeffect can be obtained by providing a current frame rate setting deviceinstead of the frame rate detector 45.

Furthermore, the frame rate detector 45 of the third embodiment detectsthe frame rate of the moving-image data output from the image pickupdevice 1 or another moving-image input stage. Because the synthesiscontroller 4 controls the reading of the image-data-for-synthesis G2Gfrom the data-for-synthesis storage 3 in accordance with the frame ratedetected by the frame rate detector 45, the motion of theimage-data-for-synthesis G2G stored in the data-for-synthesis storage 3can be kept in the optimum and effective state even if the frame rate ofthe moving-image data changes.

To be more specific, the synthesis controller 4 a of the thirdembodiment controls the reading of the image-data-for-synthesis G2G tothe data-for-synthesis storage 3 in such a manner that the repetitioncount of the image-data-for-synthesis G2G becomes M when the frame rateof the moving-image data output from the image pickup device 1 oranother moving-image input stage is N*M, and the repetition count of theimage-data-for-synthesis G2G from the data-for-synthesis storage 3becomes L when the frame rate becomes N*L. According to the thirdembodiment, the motion of the image-data-for-synthesis G2G stored in thedata-for-synthesis storage 3 is kept in the optimum and effective state,or the state of frame rate N, even if the frame rate of the moving-imagedata changes.

Except for the above described respects, the third embodiment is thesame as the first embodiment. Further, the synthesis controller 4 a ofthe embodiment can be applied to the moving-image synthesis device ofthe second embodiment.

MODIFICATIONS OF THE PRESENT INVENTION

The structure and the parameters of the control-data-for-synthesis(header) G2C of the embodiments of the present invention are not limitedto those shown in FIG. 4. The similar effect can be obtained even whenthe control-data-for-synthesis (header) G2C has “a horizontal startingposition of composite image”, “a vertical starting position of compositeimage”, “a horizontal ending position of composite image”, and “avertical ending position of composite image”, instead of “the horizontalposition 3111 of image-for-synthesis”, “the vertical position 3112 ofimage-for-synthesis”, “the horizontal size 3113 of image-for-synthesis”,and “the vertical size 3114 of image-for-synthesis”, or when the orderof the parameters is changed as desired.

Further, in the embodiments described above, there is no need to storethe headers G2C continuously in the control-data-for-synthesis storage31 as shown in the figure or to store the items ofimage-data-for-synthesis G2G continuously in theimage-data-for-synthesis storage 32.

Furthermore, in the first embodiment and the third embodiment, thecontrol-data-for-synthesis storage 31 and the image-data-for-synthesisstorage 32 are separately provided in the data-for-synthesis storage 3.However, the control-data-for-synthesis (header) G2C and theimage-data-for-synthesis G2G may be stored in a united state or in amixed state.

Moreover, in the embodiments described above, when a sequence is formed,there is no need to match the order of the items ofcontrol-data-for-synthesis (header) G2C stored in thecontrol-data-for-synthesis storage 31 with the order of the items ofcontrol-data-for-synthesis (header) G2C forming the sequence. Similarly,the items of image-data-for-synthesis G2G stored in theimage-data-for-synthesis storage 32 may be stored in any order.

1. A moving-image synthesis device comprising: a synthesis processorwhich receives a video signal, which includes moving-image data and amoving-image control signal including display timing information of eachframe of the moving-image data; and a storage which storesdata-for-synthesis, which includes a plurality of items ofimage-data-for-synthesis and a plurality of items ofcontrol-data-for-synthesis associated with the plurality of items of theimage-data-for-synthesis: wherein the synthesis processor reads one ofthe plurality of items of the control-data-for-synthesis from thestorage at a timing based on the moving-image control signal asprocessing A; reads the image-data-for-synthesis associated with theread control-data-for-synthesis from the storage in accordance with theread control-data-for-synthesis as processing B; executes processing tosynthesize one frame of the moving-image data and the readimage-data-for-synthesis as processing C; and repeats the processing Athrough the processing C, thereby producing composite moving-image dataas processing D.
 2. The moving-image synthesis device according to claim1, wherein if the read control-data-for-synthesis includes repetitioncount information specifying that the processing B and the processing Cbe repeated more than once, the synthesis processor repeats theprocessing B and the processing C for the repetition count beforeexecuting the processing D.
 3. The moving-image synthesis deviceaccording to claim 1, wherein each of the items of thecontrol-data-for-synthesis stored in the storage includes pointerinformation indicating an item of the control-data-for-synthesis to beused for next synthesis processing; and in the processing A through theprocessing C repeated in the processing D, thecontrol-data-for-synthesis read from the storage is the item of thecontrol-data-for-synthesis indicated by the pointer information.
 4. Themoving-image synthesis device according to claim 1, wherein each of theitems of the control-data-for-synthesis stored in the storage includesdisplay position information and display size information of theimage-data-for-synthesis associated with the control-data-for-synthesis;and in the processing C, the synthesis processor overlays animage-for-synthesis of a size based on the display size information in aposition based on the display position information.
 5. The moving-imagesynthesis device according to claim 1, wherein the moving-image controlsignal includes information of a frame rate of the moving-image data;and the synthesis processor controls the reading of thecontrol-data-for-synthesis from the storage in accordance with the framerate.
 6. The moving-image synthesis device according to claim 1, whereinthe moving-image control signal includes information of a frame rate ofthe moving-image data; and when the frame rate of the moving-image datais N*M, where N and M are respectively positive integers, and therepetition count of the processing A through the processing C is M, ifthe frame rate is multiplied by L/M, where L is a positive integer, tobe set to N*L, the synthesis processor multiplies the repetition countof the processing A through the processing C by L/M to set therepetition count to L.
 7. The moving-image synthesis device according toclaim 1, wherein the processing C by the synthesis processor includesprocessing to attenuate amplitude levels of the moving-image data andthe image-data-for-synthesis and add the attenuated amplitude levels ofthe moving-image data and the image-data-for-synthesis.
 8. Themoving-image synthesis device according to claim 7, wherein thesynthesis processor has a function to adjust an attenuation rate of theamplitude level of the moving-image data and an attenuation rate of theamplitude level of the image-data-for-synthesis.
 9. The moving-imagesynthesis device according to claim 7, wherein the synthesis processorselectively outputs any of the moving-image data, theimage-data-for-synthesis, and the image data obtained from theprocessing of adding.
 10. A moving-image synthesis method comprising thesteps of: storing data-for-synthesis, which includes a plurality ofitems of image-data-for-synthesis and a plurality of items ofcontrol-data-for-synthesis associated with the plurality of items of theimage-data-for-synthesis, as a step A; receiving a video signal, whichincludes moving-image data and a moving-image control signal includingdisplay timing information of each frame of the moving-image data, as astep B; reading one of the plurality of items of the storedcontrol-data-for-synthesis at a timing based on the moving-image controlsignal, as a step C; reading the image-data-for-synthesis associatedwith the read control-data-for-synthesis from the plurality of items ofthe stored image-data-for-synthesis in accordance with the readcontrol-data-for-synthesis, as a step D; executing processing tosynthesize one frame of the moving-image data and the readimage-data-for-synthesis, as a step E; and repeating the step C throughthe step E, thereby producing composite moving-image data, as a step F.11. The moving-image synthesis method according to claim 10, whereineach of the items of the stored control-data-for-synthesis includespointer information indicating an item of the control-data-for-synthesisto be used for next synthesis processing; the control-data-for-synthesisread in the step C through the step E repeated in the step F are thecontrol-data-for-synthesis pointed at by the pointer information. 12.The moving-image synthesis method according to claim 10, wherein themoving-image control signal includes information of a frame rate of themoving-image data; and the reading of the storedcontrol-data-for-synthesis is controlled in accordance with the framerate.
 13. The moving-image synthesis method according to claim 10,wherein the moving-image control signal includes information of a framerate of the moving-image data; and when the frame rate of themoving-image data is N*M, where N and M are respectively positiveintegers, and the repetition count of the step C through the step E isM, if the frame rate is multiplied by L/M, where L is a positiveinteger, to be set to N*L, the repetition count of the step C throughthe step E is multiplied by L/M to be set to L.
 14. The moving-imagesynthesis method according to claim 10, wherein if the readcontrol-data-for-synthesis includes repetition count informationindicating that the repetition count of the step D and the step E is oneor more, the step D and the step E are repeated for the repetition countbefore the step F is executed.
 15. The moving-image synthesis methodaccording to claim 10, wherein the step E includes processing toattenuate amplitude levels of the moving-image data and theimage-data-for-synthesis and add the attenuated amplitude levels of themoving-image data and the image-data-for-synthesis.
 16. An informationterminal apparatus with a moving-image synthesis function, comprising:an image pickup device which generates a video signal, which includesmoving-image data and a moving-image control signal including displaytiming information of each frame of the moving-image data; a synthesisprocessor which receives the video signal; a storage which storesdata-for-synthesis, which includes a plurality of items ofimage-data-for-synthesis and a plurality of items ofcontrol-data-for-synthesis associated with the plurality of items of theimage-data-for-synthesis; and a videophone processor which has afunction to send composite moving-image data; wherein the synthesisprocessor reads one of the plurality of items of thecontrol-data-for-synthesis from the storage at a timing based on themoving-image control signal as processing A; reads theimage-data-for-synthesis associated with the readcontrol-data-for-synthesis from the storage in accordance with the readcontrol-data-for-synthesis as processing B; executes processing tosynthesize one frame of the moving-image data and the readimage-data-for-synthesis as processing C; and repeats the processing Athrough the processing C, thereby producing composite moving-image dataas processing D.
 17. The information terminal apparatus with themoving-image synthesis function according to claim 16, furthercomprising a data-for-synthesis input section for supplying the storagewith the data-for-synthesis.
 18. An information terminal apparatus witha moving-image synthesis function, comprising: a video signal inputsection which receives a video signal, which includes moving-image dataand a moving-image control signal including display timing informationof each frame of the moving-image data; a synthesis processor whichreceives the video signal; a storage which stores data-for-synthesis,which includes a plurality of items of image-data-for-synthesis and aplurality of items of control-data-for-synthesis associated with theplurality of items of the image-data-for-synthesis; and an image displaysection which displays an image based on composite moving-image data;wherein the synthesis processor reads one of the plurality of items ofthe control-data-for-synthesis from the storage at a timing based on themoving-image control signal as processing A; reads theimage-data-for-synthesis associated with the readcontrol-data-for-synthesis from the storage in accordance with the readcontrol-data-for-synthesis as processing B; executes processing tosynthesize one frame of the moving-image data and the readimage-data-for-synthesis as processing C; and repeats the processing Athrough the processing C, thereby producing composite moving-image dataas processing D.
 19. The information terminal apparatus with themoving-image synthesis function according to claim 18, furthercomprising a data-for-synthesis input section for supplying the storagewith the data-for-synthesis.